Fix weight loading issue #14016
Merged
Fix weight loading issue #14016
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The issue in #14002 comes from the fact, when using However, it is required to extend the variable scope for the 2 components (encoder & decoder) of a TF composite models - to make the subsequent I feel it would be better to modify transformers/src/transformers/modeling_tf_utils.py Lines 1445 to 1449 in d5b82bb |
|
Looks great to me! |
patrickvonplaten
approved these changes
Oct 20, 2021
|
Hi, I just realized that |
ydshieh
force-pushed
the
fix_tf_enc_dec_weight_loading
branch
from
412bfed to
4a5f899
Compare
patrickvonplaten
added a commit
that referenced
this pull request
Nov 15, 2021
This reverts commit a67d47b.
ydshieh
added a commit
to ydshieh/transformers
that referenced
this pull request
Dec 26, 2021
ydshieh
added a commit
to ydshieh/transformers
that referenced
this pull request
Jan 6, 2022
sgugger
added a commit
that referenced
this pull request
Jan 10, 2022
* Start the work on TFVisionEncoderDecoderModel * Expose TFVisionEncoderDecoderModel * fix import * Add modeling_tf_vision_encoder_decoder to _ignore_modules in get_model_modules() * reorder * Apply the fix for checkpoint loading as in #14016 * remove attention_mask + fix VISION_DUMMY_INPUTS * A minimal change to make TF generate() work for vision models as encoder in encoder-decoder setting * fix wrong condition: shape_list(input_ids) == 2 * add tests * use personal TFViTModel checkpoint (for now) * Add equivalence tests + projection layer * style * make sure projection layer can run * Add examples * Apply suggestions from code review Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * Clean comments (need to work on TODOs for PyTorch models) * Remove TF -> PT in check_pt_tf_equivalence for TFVisionEncoderDecoderModel * fixes * Revert changes in PT code. * Update tests/test_modeling_tf_vision_encoder_decoder.py Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * Add test_inference_coco_en for TF test * fix quality * fix name * build doc * add main_input_name * Fix ckpt name in test * fix diff between master and this PR * fix doc * fix style and quality * fix missing doc * fix labels handling * Delete auto.rst * Add the changes done in #14016 * fix prefix * Apply suggestions from code review Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * make style Co-authored-by: ydshieh <ydshieh@users.noreply.github.com> Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fix #14002 + add 2 more tests
(I uploaded the converted TF checkpoint to ydshieh). Might be better for @patrickvonplaten to upload to patrickvonplaten
Who can review?
@Rocketknight1 @patrickvonplaten @LysandreJik